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Advanced Placement Exam Scores 
as a Predictor of Performance in 
Introductory College Biology, 
Chemistry and Physies Courses 

Survey data from 8,594 students in 55 randomly chosen colleges and universi- 
ties finds that those having passed an AP science exam earn somewhat higher 
college science grades, but not enough to assume prior mastery. Moreover, 
half of this performance difference appears to be related to demographics 
and high school coursework and not to students' AP coursework. 


Introduction 

Students who take Advanced 
Placement (AP) examinations in high 
school seek the option of receiving 
credit for and bypassing introductory 
college level courses. These exams are 
constructed and graded hy the College 
Entrance Examination Board (College 
Board) to “allow participating col- 
leges and universities to award high 
school students college credit, ad- 
vanced placement, or both ” ( College 
Board, 2004, p. 6). While the granting 
of credit or placement remains at the 
discretion of each individual college 
or university, the College Board is 
unequivocal in its interpretation of 
AP exam scores: 

AP Exam grades of 5 are 
equivalent to the top A-level 
work in the corresponding 
college course. AP Exam grades 
of 4 are equivalent to a range 
of work representing mid-level 
A to mid-level B performance 
in college. Similarly, AP Exam 


grades of 3 are equivalent to 
a range of work representing 
mid-level B to mid-level C 
performance in college (College 
Board, 2006, p. 1).' 

The purpose of this study is to ap- 
praise the College Board’s claim of the 
equivalence of Advanced Placement 
courses as measured hy AP exam 
scores with the comparable college 
course in three science disciplines: 
biology, chemistry, and physics. This 
is a complex issue because, to some 
degree, AP course-taking is a sorting 
process that draws primarily highly 
motivated, high achieving students. 
If this self- selection effect is ignored. 



Colleges vary considerably 
in their policies and 
in their advice to their 
students concerning 
enrollment in introductory 
science courses. 


the outcome of AP course-taking itself 
may be overestimated. Dougherty, 
Mellor and Jian (2006a) offered the 
following insight: 

Much of those [AP] students 
later success in college may 
be due not to the AP classes 
themselves, but to the personal 
characteristics that led them 
to participate in the classes 
in the first place - better 
academic preparation, stronger 
motivation, better family 
advantages, and so on. These 
selection effects will affect any 
comparison of AP and non-AP 
students (p. 3). 

Colleges vary considerably in their 
policies and in their advice to their 
students concerning enrollment in 
introductory science courses. As a 
result, there exists a sizable population 
of students taking introductory science 
courses who have done well in AP 
science courses while in high school. 


l.In this research we refer to the numerical score earned on the Advanced Placement exam (1, 2, 3, 4, or 5) as the “AP exam score.” We refer to the teacher’s as- 
sessment of the AP students’ coursework as the “AP course grade.” In its literature, the College Board generally refers to the AP exam score as the “AP grade.” 
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yet do not “place out” in college. Since 
the AP exam is taken prior to college 
entry, one would expect that these 
students, whom the College Board has 
awarded passing exam scores, would 
earn suhstantially higher grades , on the 
average, upon re-taking an introduc- 
tory college course in the same sub- 
ject. The apparent advantage is clear. 
Students exhibiting higher levels of 
AP performance would most certainly 
be expected to have a stronger grasp 
of the relevant college content. They 
should do better than students who 
have not had the advantage of what 
is typically a second year of study in 
the subject while in high school. The 
intent of this paper is to characterize the 
performance of these students as both 
an independent approach to the valida- 
tion of Advanced Placement exams in 
biology, chemistry, and physics and as 
an estimate of the contribution of these 
advanced high school courses. 

Background 

The AP Program of the College 
Board provides “motivated high 
school students with the opportunity 
to take college-level courses in a high 
school setting.”^ High school AP 
courses corresponding to the 35 AP 
examinations administered by the 
College Board in both science and 
non-science disciplines are taught by 
110,000 teachers in the US. These 
exams are central to the AP Program 



High school AP courses 
corresponding to the 35 AP 
examinations administered 
by the College Board in both 
science and non-science 
disciplines are taught by 
110,000 teachers in the US. 

and offer a means of certifying each 
student’s mastery of course content at 
the college level. In the sciences, stu- 
dents sat for a total of 3 1 1 ,595 exams 
in environmental science, biology, 
chemistry, or physics in 2005 . ^ Among 
these science test-takers, 59% earned 
scores of 3 or higher (5-point scale). 
By comparison, 62% of mathematics 
test-takers earned scores of 3 or better. 
As a result, many of these students 
would have grounds to expect their 
prospective colleges to offer them 
course credit in these disciplines when 
they matriculate.'* This outcome would 
be consistent with the original purpose 
of Advanced Placement envisioned 
by representatives of the faculties of 
the three private academies (Andover, 
Exeter, and Lawrenceville) and 
three private universities (Harvard, 
Princeton, and Yale) responsible for 
formulating the program. ^ To promote 
this policy, the College Board makes 
available research studies that support 
its contentions.® 


With tests scores in hand, students 
are assured by the College Board 
that “Over 90 percent of the nation’s 
colleges and universities have an AP 
policy granting incoming students 
credit, placement, or both, for qualify- 
ing AP Exam grades.”^ However, the 
fine print is more complex. Having 
an “AP policy” does not mean that 
colleges and universities follow the 
College Board’s recommendation that 
an AP score of 3 should gamer course 
credit or advanced placement. Many 
times, colleges only offer credit for 
scores of 4 or even 5. Some college 
policies do not grant either to incom- 
ing students.® 

While the College B oard puts much 
energy into the construction, monitor- 
ing, and scoring of AP exams in an 
attempt to attain a measure of college 
course equivalency, there is a consider- 
able range of opinion concerning the 
value of AP courses in the sciences and 
mathematics. Many professors believe 
that all students can benefit from taking 
introductory science courses in college 
(National Research Council, 2002, p. 
58). Others argue that college-level 
courses require the efforts of college- 
level faculty (Juillerat, Dubowsky, 
Ridenour, McIntosh & Caprio, 1997). 
Some colleges administer their own 
placement exams. Others allow credit 
only for students not majoring in that 
specific field or award credit only after 
a student has taken and passed a higher- 


2. College Board AP Central website, http://apcentral.collegeboard.com. 

3. http://apcentral.collegeboard.com/members/article/l,3046, 152-17 1-0-39036, 00. html 

4. “ACE [American Council on Education] notes that the practice of granting provisional credit for a grade of 2 is becoming more frequent because comparabil- 
ity studies have shown that many of these students are potentially qualified for college-level work — credit is made permanent if the student satisfies another 
criterion, such as successfully completing the next course in the discipline.” http://apcentral.collegeboard.com/apc/public/colleges/setting_policy/index.html 

5. “[For] the student of marked mathematical and scientific ability and interest, who is probably heading for a college major in science and a career in science or 
engineering . . . could take a one-and-a-half or two-year school course in chemistry, physics, or biology, designed to prepare him for admission to the sopho- 
more college course in the field chosen for concentration in school [i.e. in college]. (Phillips Academy, 1952, p. 65.) 

6. http://apcentral.collegeboard. com/article/0,3045, 152-167 -0-1 1592,00. html 

However, research that can be considered somewhat critical of the claims of the College Board concerning the AP program does not appear to be included at 
their website (e.g. Geiser & Santelices, 2004; Lichten, 2000). 

7. http://apcentral.collegeboard.com/program/ 0,3060, 150-0-0-0,00. html 

8. Lichten’s (2007) sampling of colleges and universities found that only 30% of colleges accept an AP exam score of 3 for advanced placement, while 89% 
accept a score of 4, and 95% accept a score of 5. 
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level course in the subject (National 
Research Council, 2002, p. 58). Some 
universities offer special transitional 
courses for those students who place 
out of introductory courses, but do 
not advance to the next level course 
(National Research Council, 2002, p. 
59).^ Lichen (2000) argues that the 
quality of AP courses has diminished 
to the point that the College Board’s 
recommendations are no longer ac- 
cepted by a large number of colleges, 
particularly those considered highly 
selective (e.g. Harvard, Yale, MIT, 
University of Pennsylvania). An 
NRC committee recommends that col- 
leges should be strongly discouraged 
“. . .from using college scores onAPan 
IB exams as the sole basis for granting 
automatic advanced placement out of 
specific courses for majors, or out of 
biology distribution requirements for 
non majors (Wood 2002, p. 125.)” 
The population of students who 
have taken AP courses in high school 
and retake introductory courses has 
been has been largely neglected by re- 
searchers." Surprisingly, researchers 
have rarely structured past studies to 
reveal the degree to which AP courses 
bestow an added benefit upon students 
who take them, “ . . . the existing em- 
pirical evidence regarding the benefits 
of AP experience is questionable 
(Klopfenstein and Thomas, 2005, p. 
5).” Klopfenstein and Thomas refer to 
the body of AP research, a majority of 
which lack or include insufficient con- 
trols. We should keep in mind that, in 


addition to attracting potentially more 
highly motivated and gifted students, 
there are other potential covariates: 
AP courses are more prevalent in 
wealthier communities, are typically 
taught by more experienced teachers 
(Burdman, 2000) and predominantly 
enroll students who have already 
studied the subject for a year. Much 
of the existing AP research gives no 
recognition to these factors. Studies 
lacking controls suffer the potential 
weakness of overestimating the effects 
of AP courses, because they conflate 
the influence of numerous factors and 
fail to account for the relationship of 
AP courses net of the co variates. While 
many assume that students who do 
well in AP courses in high school owe 
their success to their AP studies, other 
factors may well be involved, such as 
coming from more advantaged homes 
and attending schools that emphasize 
college preparation (Willingham & 
Morris 1986). Moreover, the fast pace 
of AP courses, due in large measure 
to the focus on test preparation, may 
not serve students as well as courses 
that progress at a more moderate pace, 
focusing on topics in greater depth and 
detail. This supposition led a recent na- 
tional study to posit that, “It is possible 
that AP students were at a disadvantage 
in some classes or at some colleges” 
(National Research Council, 2002, p. 
194). While most assume that such an 
outcome is unlikely, we are willing to 
subject such hypotheses to scrutiny. 
For example, students reporting the 


lowest score of 1 (out of 5) on the AP 
exam have not demonstrated content 
mastery. However, this outcome leads 
one to wonder whether this year for 
these students of study could have been 
better spent in a course with a differ- 
ent format. Also troubling are the low 
AP science exam passing rates among 
under-represented groups.'^ 

Prior Research 

Research on the value of AP 
courses in the sciences has primarily 
focused on accounting for differences 
in freshman grade point average or 
persistence to graduation, both rather 
indirect measures of the equivalence 
of AP courses and introductory col- 
lege science courses. Others judge the 
performance of students who “place 
out of’ one or two semesters of the 
introductory course in a scientific field 
and then enroll in a second or third 
semester science course equivalent 
to students who take the prerequisite 
course in college. Such studies have 
not controlled fully for the fact that 
AP students generally have stronger 
high school preparation in math and 
English, contributing to the ability to 
“skip ahead.” Moreover, second and 
third semester science courses can 
cover content unrelated to the pre- 
requisite course (e.g. second semester 
physics usually entails a study of elec- 
tricity, magnetism, and light, topics 
distant from first semester kinematics 
and Newtonian mechanics). In most 
cases, studies examine the perfor- 


9. None of the courses participating in this study have that profile. 

10. USA Today, 3/20/2006, Advanced Placement: A detour for college fast track? Mary Beth Marklein 

11. Studies comparing student performance in second-year college courses typically compare those who have “placed out” with high AP exam scores to those 
with students who take the introductory level college course with either a failing AP exam score or no AP experience (Dodd, Fitzpatrick, De Ayala, & Jen- 
nings, 2002). No mention is found in the literature of the experience of those students who chose to retake the introductory college course with a passing AP 
exam score. 

12. Fewer than half of under-represented minorities are “passing” the AP exam with scores of 3 or higher, while scores of 1 (the lowest score possible) are 
awarded to more than half of African American and Chicano students. Put in another way, students not “passing” the exam constitute 70% of Chicano and 
African-American students, while more than 50% of Native Americans and Puerto Rican students score below 3. Based on this measure, AP courses do not 
appear to be an effective means of AP exam preparation for many students. For more information, please consult the AP National Report 2005, http://www. 
collegeboard.com/student/testing/ap/exgrd_sum/2005. html 
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The elan of having AP 
courses, especially in 
science and mathematics, 
is well known. 

mance of students at a single college 
or university or within a single state, 
making generalization to the entire 
U.S. problematic. These studies also 
fall into two general categories : studies 
with adequate control variables and 
those without. The latter group typi- 
cally compared grand means between 
samples of AP and non- AP students or 
correlated the performance of AP stu- 
dents with other performance-related 
variables. We found that these studies 
typically appear as reports that are not 
published in peer-reviewed journals. 
Often the source of funding for much 
of this work is unclear. On the other 
hand, the “with controls” studies more 
frequently appear in peer-reviewed 
journals, having been subjected to 
more scrutiny and review. 

While we reviewed both forms of 
research, we summarized only the 
findings of the “with-controls” studies 
below, grouping the findings under 
four commonly analyzed outcome 
variables. The 1986 Willingham and 
Morris report was especially useful in 
constructing this list. 

• College Grade Point Average 

(GPA)- The total number of AP 
courses taken in high school has 
little effect on predicting fresh- 
man college GPA, but AP exam 
scores are a strong predictor of 
sophomore grades (Geiser & 
Santelices, 2004). Using matched 
sets totaling 688 students at 
Indiana University, AP students 
were found to earn higher college 
GPA’s (Chamberlain, Pugh, & 


Shellhammer, 1978). Willingham 
and Morris’ (1986) found that 
about half of the grade differen- 
tial (those with a college grade 
average of B or above) of AP 
students over non-AP students 
was accounted for by matching 
pairs of 1878 students on their 
academic background. Klopfen- 
stein & Thomas (2005) found that 
of 28,167 students, those taking 
AP courses in high school did 
not earn higher freshman grades 
in college. 

• College Science Course Grades. 

Ruch (1968) did not find a sta- 
tistically significant difference 
between matched pairs of AP 
and non-AP students on college 
grade in the courses related to the 
AP course taken or overall fresh- 
man GPA. Dodd, Fitzpatrick, 
and Jennings (2002) studied the 
performance of three groups of 
83 1 students in a second semester 
biology course at the University 
of Texas at Austin. They compare 
the mean grades of three student 
groupings: 

1 . Those who earned credit for the 
prerequisite course with a score 
of 3 or above on their AP exam 
(AP-CR). 

2. Those who earned lower than 
a 3 in their AP course and took 
the prerequisite (AP-Class). 

3. Those who did not take an AP 
class and enrolled in the perqui- 
site class (Non-AP). 

AP-CR and Non-AP groups were 
matched using high school class rank 
and SAT and ACT scores. Over a 
four-year period, the Non-AP group 
earned higher mean grades than either 
the AP-Class or the AP-CR groups 
in the second semester course. This 


difference was reported to be statisti- 
cally significant for one year of the 
study. The AP-Class group earned 
slighted higher grades in the second 
semester course than the AP-CR 
group. Reported GPA’s in other college 
biology courses were also higher for 
Non-AP students. 

• Persistence to college gradua- 
tion - The combined number of 
AP and honors courses taken in 
high school was not a significant 
predictor for persistence through 
freshman to sophomore years in 
college (Geiser & Santelices, 
2004); Klopfenstein & Thomas 
(2005) found students taking AP 
courses in high school have no 
higher probability of persisting 
to a second year of college. Low- 
income students who take AP 
courses in high school graduate 
college at a greater rate (Dough- 
erty, Mellor & Shuling, 2006a) 

• Choice ofFurther Study -Ruch 
(1968) used 21 matched-pairs 
of students and found that that 
AP students were more likely to 
continue with the same subject 
in college. Chamberlain, Pugh, 
& Shellhammer (1978) found 
academic progress, college GPA, 
and course-taking measures were 
superior for AP students over 344 
matched sets of students (Cahow, 
Christensen, Gregg, Nathans, 
Strobel, & Williams, 1979). Will- 
ingham and Morris ’(1986) found 
thatof 1878 students, AP students 
were no more likely to take more 
than one year’s course work in 
biological or physical science, 
nor to major in those fields. 

All of the studies above controlled 
for students’ high school performance 
(i.e. rank, HSGPA, or coursework 
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completed) except for Chamberlain 
et al. (1978) and Dougherty et al. 
(2006a). Geiser & Santelices (2004), 
Klopfenstein & Thomas (2005), and 
Dougherty et al. (2006a) included 
measures of students’ socio-economic 
status. Ruch (1968), Dougherty et al. 
(2006a) and Klopfenstein & Thomas 
(2005) included high school quality 
ratings, while Willingham & Morris 
(1986) matched students by college 
attended. 

From this short list of controlled 
studies, it is apparent that the measure 
of AP course work value varies with the 
outcome variable considered and the 
control variables included. We did not 
find any studies that measured perfor- 
mance in introductory college science 
courses while controlling for relevant 
CO variates such as performance in high 
school courses considered prerequi- 
sites for AP science courses. Given 
that performance in prior coursework 
and is widely regarded as the strongest 
predictor of subsequent performance, 
it is essential that this factor should 
be included in any analysis seek- 
ing to isolate the relationship to AP 
coursework itself, rather than merely 
considering an aggregated analysis 
that lacks both detail and is unable 
to estimate the relative contributions 
of various factors. The inclusion of 
such controls provides an opportunity 
for refinement of models that more 
accurately estimate the relationship 
of AP coursework relative to other 
factors. Regression models must be 
constructed using variables represent- 
ing viable alternative hypotheses to 
the main relationship being studied 


in order to be considered robust. For 
example, inclusion of SAT scores 
alone, entered as a control for high 
school-level learning, is insufficient 
to account for prior science knowl- 
edge learned in first-year high school 
science courses. It is incumbent on 
the scholar employing regression to 
explore alternative relationships that 
are potentially causal and to take great 
care in considering, measuring, and 
including factors that could account 
for additional variance. Put another 
way, it is the attention that scholars 
pay to controlling for alternative 
hypotheses that is often a measure of 
quality research. 

But, why is research on the direct 
comparison of AP and non- AP students 
important? Conventional wisdom 
ascribes clear and overwhelming ben- 
efits to Advanced Placement classes 
in high schools. The elan of having 
AP courses, especially in science and 
mathematics, is well known. As a re- 
sult, the College Board’s AP program 
has grown dramatically over the last 
20 years, with a sustained increase 
in the number of exams including all 
subject areas of 9.3% (with an 8.8% 
increase in number of students) annu- 
ally. To offer some perspective, this 
growth is more than five times faster 
than the growth in the US high school 
population. The ratio of students to 
exams tracked closely until recently; 
their separation is indicative of an 
increase in the number of students tak- 
ing multiple AP exams (1.29 in 1970 
vs. 1.72 in 2005). In 2005, 1.2 million 
students took 2.1 million AP exams. 
According to the latest available data. 


2. 1 million entering freshmen enrolled 
in degree granting institutions in Fall 
2003. One might assume that on the 
average nearly half of all entering col- 
lege freshmen have taken at least one 
AP course in high school. In reality, 
many have taken more than one course. 
Our estimate is that in 2004, those 
taking AP science exams represented 
one of five college freshmen (Digest 
of Educational Statistics, 2004).'^ 

The expansion of AP Programs is 
also currently propelled by economic 
and political support since several 
states currently extend financial in- 
centives to high schools that offer AP 
courses'®, while others require all high 
schools to participate (Hershey, 1990; 
Willingham & Morris, 1986). Many 
states have mandated through legisla- 
tion the inclusion of AP enrollment in 
admissions decisions or the granting 
of college credit for high AP exam 
scores in public institutions of higher 
education (Lichten, 2000). The U.S. 
president has called for an addition of 
70,000 Advanced Placement teachers 
in science and mathematics, a near 
doubling of current numbers. In addi- 
tion, the U.S. Department of Education 
subsidizes AP programs for low-in- 
come students (Klopfenstein, 2004).'^ 
This expenditure of public funds, in 
effect, subsidizes one particular edu- 
cational program over others. Hence, it 
is desirable that Advanced Placement 
policies and programs be rigorously 
assessed so that scarce resources can 
be spent the most wisely. 

A powerful motivating factor for 
students is the fact that one may enter 
college with course credits and in some 


13. http://apcentral.collegeboard.com/members/article/l,3046, 152-17 1-0-47040 , 00. html. 

14. http://nces.ed.gOv/programs/digest/d05/tables/dt05_180.asp. 

15. http://apcentral.collegeboard.com/members/article/l,3046, 152-17 1-0-39036, 00. html. 

16. Florida, Louisiana, and Utah. 

17. South Carolina. 

18. George W. Bush, State of the Union Address, 1/31/2006. 

1 9. Higher Education Act Amendments of 1998, Title VIII, Part B, P. L. 105-244. 
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cases with enough credits to bypass the 
freshman year, saving a year’s tuition 
(MacVicar, 1988; Pushkin, 1995). 
Troubling to many professors is that 
students may use AP credit in many 
colleges to satisfy their requirements 
for science courses and never take 
another science course when in college 
(National Research Council, 2002, 
p. 59). For example, Lichten (2000) 
found that only 22% of students who 
had scored “3” on their AP calculus 
exam enrolled in an advanced calculus 
course in college, while 17% took a 
remedial calculus course. Though 
high school AP teachers may be very 
talented, they generally do not have 
the breadth of knowledge nor the most 
recent and relevant content knowledge 
possessed by college faculty (Lichten 
2000). By accepting AP exam-related 
course credits, students may be forego- 
ing the opportunity to learn about the 
most topical findings and issues from 
college instructors whose “other” job 
it is to do research. 

The striking expansion of the AP 
program offers an opportunity to 
gauge the pedagogical influence of 
AP coursework. Merely presuming 
that AP courses are comparable with 
college courses ignores the opportu- 
nity we have to analyze the degree of 
this purported similarity as a means to 
improve college preparation through 
high school instruction. 

Methods 

The Sample 

Factors Influencing College Science 
Success (Project FICSS) is a large- 
scale national study of introductory 
college students and their high school 
science experiences, which collected a 
total of nearly 1 8 ,000 surveys. The data 
includes a series of subsamples of col- 
lege students in various college science 
courses. In this particular analysis, we 


analyzed survey data gathered from 
students in 124 different first semester 
introductory college biology, chemis- 
try, and physics courses in 55 colleges 
and universities. These institutions 
were from a stratified random sample 
reflecting a nationally representative 
distribution of institutional size and 
selectivity, from small liberal arts 
colleges to large state universities. 
All participating introductory science 
courses met in lecture-style classes 
with weekly recitations sessions and 
laboratories and had enrollments of 
10 students or more. 

Of 9354 collected surveys in this 
subsample, 1029 respondents reported 
having taken an AP course in the sub- 
ject in high school in which they were 
currently enrolled in college. Of the 
students who took AP courses, 316 
earned scores of 3 or higher on the 
AP exam, roughly 3% of the entire 
sample. While this can be considered 
a small number compared to the 
170,000 students who earned scores 
of 3 or higher on AP exams in these 
subjects in 2005, the data used in this 



... many students feel that while 
AP is a good preparation for 
college science, many also feel 
they benefited from taking the 
actual college science course. 

analysis represents a national sample 
of students who have both taken part 
in Advanced Placement courses and 
exams in high school and also take 
introductory level college science 
courses. The theory and practice of 
inferential statistics has established 
that valid inferences (within ap- 
propriate confidence limits) can be 
drawn from relatively small number 


of participants, provided they are from 
a well-selected random sample. 

Choosing to focus only on under- 
graduates, we omitted cases of 103 
graduates and 400 non-traditional 
students. We also omitted 397 students 
who spent their high school years 
outside of the United States. Allowing 
for some students in the intersection of 
these various groups, our final sample 
included 8594 undergraduates. 

Of this total, 3 1 % of the sample was 
enrolled in biology, 42% in chemistry, 
and 27 % in physics. Females made up 
60% of the biology students, 55% of 
the chemistry, and 42% of the physics 
students. Parental education was relat- 
ed to the highest level of mathematics 
taken by students, with representation 
of students taking calculus in high 
school ranging from 33% of student 
with parents having a high school di- 
ploma or less to 49% of students with a 
parent holding a graduate degree. The 
sample identified themselves as 77% 
White, 7% Asian/Pacific Islander, 6% 
black, 5% Hispanic, 3% multi-racial, 
and 2% other or unreported. 

This study, by its very nature, may 
only examine students who take intro- 
ductory-level college science courses. 
Students enrolled in colleges that offer 
the option of bypassing introductory- 
level science courses with sufficient 
AP exam scores and who also make 
the decision to take advantage of this 
option to place out are not included. 
Yet, since prior research studies have 
generally ignored this interesting 
population, studying these students 
can make a valuable contribution to 
understanding the value of Advanced 
Placement programs. 

Students who did poorly on AP 
exams in a given science discipline 
may choose not to take the college 
course. By comparing the national 
statistics for AP exam scores with the 
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Figure 1: Distribution of AP Exam Scores in Sample vs. National Norms 



—♦—Sample 
—^National Norms 


distribution in our sample, one can see 
differences (Figure l).^°The five AP 
score categories each average about 
one-fifth of the students who take the 
exam.^' It appears that students at the 
low end (AP exam scores of 1 and 2) 
and at the high end (AP exam scores 
of 4 and 5) are under-represented 
in the sample. Presumably a larger 
fraction of those who scored below 3 
chose not to enroll in a college course. 
Many at the higher end arguably took 
advantage of placement to enter a 
higher-level course or received credit 
for the introductory course. The mean 
AP exam score for the national data is 
slightly lower (2.81) than the sample 
used in this analysis (2.99) reflecting 


a relative dearth of students earning 
low AP exam scores. 

We were also careful to check for 
any anomalies in the makeup of stu- 
dents who earned scores of 3, 4, or 5 
on their AP exams. Table 1 displays 
a comparison across seven measures 
of academic performance among 
nine high school science preparation 
categories ranging from Computed 
grades represent the mean grade for a 
particular group (on a 100-point scale 
where A=95, B=85, etc.). “Not Taken” 
for students reporting that they had 
not taken a high school course in the 
discipline of their introductory college 
science course to “AP 5” for students 
reporting they had taken the Advanced 


Placement exam in the discipline of 
their introductory college science 
course and received a score of 5.^^ In 
general, AP students outperformed 
honors students, who in turn outper- 
formed regular students, who in turn 
outperformed students with no high 
school coursework in the correspond- 
ing college science discipline. Within 
the AP group, we see a clear pattern of 
higher performance corresponding to 
higher AP exam scores. High scoring 
AP 4 and AP 5 students consistently 
reported higher SAT scores, math- 
ematics grades, in-discipline science 
grades for regular or honors courses, 
and grades in other science courses. 
Higher scoring students in our sample 


20. The national statistics for AP biology, chemistry, and physics have been averaged. 

2 1 . One should note that the distribution of scores for the AP exam is not Gaussian as is reported for most national standardized exams such as the SAT or ACT. 
While those at the center of the distribution can be thought of as earning close to the mean score, those at the 4 level or above (35%) or at the 5 level (16%) 
occur with much greater frequency than students at one standard deviation above (16%) or two standard deviations above (2%). 

22. Regular signifies that the highest level of course taken in high school was neither AP nor honors level. Honors signifies an honors course. AP courses are 
broken down by AP exam scores of 1 through 5 or the category of students not taking an AP examination. Means are reported for the SAT math or SAT verbal 
scores (or the equivalent from the ACT), the fraction completing a calculus course in high school, the last grade in mathematics, the mean grade in the relevant 
science preparatory course (if taken), the mean grade in the AP course (if taken), and the average of science grades in fields other than the college science 
course of interest. 
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also reported higher grades in their AP 
courses, on average. Students in our 
sample who chose not to take the AP 
exam, even after taking the AP course , 
reported academic performance mea- 
sures similar to the students in the AP 
3 category. 

In our analysis, we were particu- 
larly concerned with the academic 
performance in college of students who 
reported receiving AP Exam scores of 
3, 4, or 5. We wondered if the students 
in our sample were particularly weak in 
these science disciplines despite their 
high AP exam scores. Might a weak- 
ness in their academic background 
have led them to pursue additional 
course work at the introductory col- 
lege level? We imagined that students 
with high AP exams, hut relatively 
weak academic backgrounds may 
not be fully fluent in English, or may 
have weaker math backgrounds, or 
may have earned lower overall high 
school grades. To address these pos- 
sibilities, we analyzed these various 
background measures and did not And 
that, compared with other students, 
high AP exam scorers exhibited any 
weaknesses in language proflciency 
(SAT Verbal score), mathematics 
proflciency (SAT Math score, high 
calculus course taking patterns, high 
school mathematics course grades), or 
overall high school grades (grades in 


various high school science courses 
including AP, regular, or honors). On 
the contrary, we found in our sample 
that students who reported an AP 
exam score of 5 also averaged 1300 
on their SAT’s, with three-quarters 
having taken calculus in high school, 
and also reported earning an “A” on 
average in their high school science 
and mathematics courses. 

Who are the group of students who 
have taken an AP course in high school 
and have earned an AP exam score 
greater than 3? Why are they taking an 
introductory course in college? Were 
they advised or required to take this 
course by their college? To gain a better 
understanding of possible motivations, 
we contacted 44 students who both 
scored well on AP exams and took 
introductory college science courses 
in these disciplines. These students had 
volunteered their contact information 
in response to a question asking if they 
would agree to future contacts in light 
of survey related queries. Among the 
responses, fifteen students provided 
detailed replies . While these responses 
should not be considered representa- 
tive, they do offer some insight into 
the diversity of students’ experiences 
and opinions. Eive made the personal 
decision to take the introductory course 
even though they could have placed 
out, citing advice from more advanced 


students in the held or their academic 
advisors. Eive took the course because 
of department requirements in their 
disciplines. Three reported AP exam 
scores of 4, while their college required 
a score of 5 for credit. Two reported 
low scores on departmental placement 
exams required by their colleges to 
earn course credit. All but one felt 
that their AP course prepared them 
well for introductory college science. 
However, eleven reported that taking 
the introductory college course as a 
very worthwhile experience that paid 
off for them later in their college sci- 
ence course work. Pour noted, in ret- 
rospect, after taking advanced courses , 
that they should have moved directly 
to higher-level course work. 

As one of those students noted: 

• I very much should have been 
[granted] credit. The class was 
pretty much a waste of my time. 

In contrast, others offered more 
positive reports of their college science 
course experiences: 

• It was worthwhile and I am glad 
I chose to take the intro level 
course. I recommend it to fresh- 
man coming in who face a similar 
situation. Taking the introductory 
course refreshes my memory on 
the subject, provides deeper 
insight into the subject that just 


Table 1: Comparison of High School Performance Measures by Group 


Highest HS 
Course or AP 
Score 

Sat Math 

SAT 

Verbal 

HS 

Calculus 

Last Math 
Grade 

Other 

Science 

Grades 

Grade in 
Honors or 
Regular 

AP Grade 

not taken 

475 

454 

27% 

86.0 

88.9 



regular 

529 

510 

32% 

87.7 

89.4 

88.8 


honors 

596 

564 

60% 

89.4 

90.8 

90.4 


AP, no exam 

610 

574 

70% 

89.6 

91.1 

92.4 

87.6 

AP 1 

543 

508 

47% 

85.2 

88.4 

90.0 

85.0 

AP 2 

592 

541 

54% 

90.1 

90.5 

91.5 

88.1 

AP 3 

595 

568 

61% 

90.1 

90.9 

91.8 

88.9 

AP 4 

631 

628 

64% 

91.0 

92.1 

93.9 

92.2 

AP 5 

666 

654 

75% 

92.5 

93.0 

94.0 

94.1 
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Figure 2: Survey Questions Relating to High School Course Taking in Science and 
Mathennatics 

D Fof Mch sdcnc* ccMirs* you took in each of the science topics listed below, please indicate the level of 
the course, the year in school you took the course, and the grade you earrted in each course. Please 
provide Information for only one addHloiral science course If this applies to you. 


SCIENCE TOPIC 
PliyjicAl Science 

COURSE LEVEL 

Honors 

o o 

YEAR IN SCHOOL 
(DCDOOO 

FINAL GRADE 

<Z)<D<S)(S><D 

BMjlogy 

o o 

(D(D900 

^<D<0(8)<D 

Chemistry 

o o 

<D(DOOO 

<X) <D <E) (B) (D 

Physics 

o o 

<D(DOOO 

<Z><D<E><B)(D 

Other 

o o 


(XMD<E><E><D 

AP Biology: APtcstKora: 01 02 03 04 OS ONotUken 

(D(Z)(900 

(Z><D<E)<S><Z> 

AP Chemistry: APtestscoie: OI 02 03 04 OS ONotUken 

(D(D900 

(Z) (D (E) (&> (D 

AP Physics: AP<AB> test scorr. OI 02 03 04 05 ONot taken 

AP(§0 test score: OI 02 03 04 05 ONotUken 

(D(D900 

<ZMD(D(S>(D 


wasn't there in high school, and 
prepares me for higher level 
courses in the subject. 

• It was definitely a worthwhile 
experience to retake the classes, 
as it gives you a second chance 
to absorb the material— which 
can never hurt. 

• I was at least familiar with many 
of the topics covered in my col- 
lege class. Many concepts were 
covered with greater detail in 
college. I felt well prepared for 
the college class, but I am glad I 
took the class instead of testing 
out. 

The student comments indicate 
that both motivations and experiences 
associated with introductory college 
science courses differ widely. 

The Survey 

The survey instrument was con- 
structed to collect data concerning aca- 
demic experiences and performance 
measures closely associated with high 
school science course work that might 
predict performance in introductory 
college science classes. In the form of 
a 4-sheet booklet, students were asked 


to complete these surveys while in their 
college classes. Using the names and 
ID numbers provided by students on 
the cover, college instructors entered 
students’ course grades. After this 
process was completed, the perforated 
cover with student identification was 
tom away and discarded, leaving the 
surveys anonymous, except for any 
contact information students may have 
volunteered in response to a “further 
contact” request. Students also re- 
ported on a variety of demographic 
variables (parents’ education, profes- 
sion, etc). Home ZIP codes allowed 
us to match the surveys with existing 
data on median income, home value, 
and average educational background 
of each home locale. The survey 
questions themselves were carefully 
designed to fit on the front and back 
of the remaining three sheets. See 
Figure 2. 

Two major concerns loom for any 
large-scale self-report study, accuracy 
and reliability. These concerns stem 
from a report by Bradburn, Rips, and 
She veil (1987) that appeared in the 
journal Science. The study concluded 
that self-report data were inaccurate 


and unreliable. However, in research 
that has followed in the intervening 
years, which includes further research 
by Bradburn himself, these conclu- 
sions have been qualified and mollified 
to portray a complex and more respon- 
sible conclusion: that the accuracy and 
reliability of self-report likely depend 
on several factors including context, 
relevance, and survey clarity Pace, 
Barahona & Kaplan, 1985; Bradburn, 
2000; Niemi & Smith, 2003). In par- 
ticular, self-reports of course taking, 
grades earned, and standardized test 
scores made by college students tend 
to be highly accurate (Baird, 1976; 
Anaya, 1999). Enrollment reports are 
especially accurate for courses with 
unambiguous names and for high- 
achieving students (Sawyer, Laing & 
Houston, 1989). In a recent review 
of existing research on self-report, 
Kuncel, Crede, & Thomas (2005) 
conclude that self-report may be 
characterized as reasonably accurate 
in samples where the surveys address 
issues relevant to the respondents. Our 
study surveyed introductory college 
science students in their fall semester 
college science classes where reflec- 
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tion on their prior experience would 
be commonplace. In addition, the 
students’ own instructors administered 
the surveys. 

We entered this project with pre- 
vious survey research experience in 
which we surveyed over 2000 college 
physics students in over 20 different 
college and universities (Sadler & 
Tai, 2001). The design of the current 
survey included the use of student 
focus groups to comment on question 
formats, two pilot surveys that in- 
cluded 304 college science students, a 
review by an advisory panel of college 
professors and high school teachers, 
and a separate reliability study where 
113 college chemistry students took 
the survey twice, two weeks apart and 
where their responses were compared. 
Much can be done to enhance accuracy 
and recall: careful design, contextual 
cues , and participant relevance all play 
important roles. 

In large-scale surveys, missing 
responses are not uncommon. In 
dealing with missing responses, list- 
wise deletion is the simplest and most 
commonly applied option. However, 
list-wise deletion assumes that the 
data are missing at random. Should a 
large percentage of the sample exhibit 
missing responses, this approach may 
not be the most appropriate option. 
We analyzed the percentage of miss- 


ing responses and found that, for the 
variables of concern, only about 1-2% 
were missing for each. Given this 
characteristic, we chose to use list- wise 
deletion. The two exceptions were 
that no standardize test score (SAT 
or ACT) was entered for 347 cases 
(quantitative) and 386 cases (qualita- 
tive). A mean value for such scores 
was substituted in these cases since 
we did not want to eliminate students 
from the dataset who had not taken 
these exams. 

Choice of controls 

Our initial analysis compares the 
mean college grades for students 
with different levels of high school 
coursework in biology, chemistry 
and physics. This particular analysis 
only shows the predictive value of AP 
exam scores on college course grades, 
quantifying the magnitude of the dif- 
ference between students who take AP 
or not, and those who score at different 
levels on the AP exam. This first step 
describes the differences that need to 
be explained by competing hypotheses 
(Dougherty, Mellor & Jian 2006a). 

However, attempting to account for 
the potential of taking an AP course 
requires controls for demographic 
and academic preparation differences 
among students as well. Far from being 
tabula rasa, students ’ skill and knowl- 
edge prior to, or unrelated to, taking an 


AP course should be included. Table 
2 shows the Pearson correlations be- 
tween academic preparation variables 
and college course grade. 

Among the nine academic perfor- 
mance measures, AP Score exhibits 
the highest correlation with College 
Grade in introductory science courses, 
though other measures with similar 
correlations include: SAT Math, 
“other” science GPA, and last high 
school math grade. A central issue 
faced by those investigating the asso- 
ciation between AP exam performance 
and college performance is that many 
other measures of academic perfor- 
mance are also positively associated 
with college grades. 

Figure 3 offers a graphical repre- 
sentation using the results produced 
from our sample. College science 
grades vary with AP exam score, 
but they also vary with the other five 
measures of academic preparation. 
As expected, more rigorous levels of 
high school science and mathematics 
courses match with higher college sci- 
ence grades. Each graph shows a clear 
monotonically increasing association 
between college grades and the six 
high school measures of academic 
achievement. It should be noted that 
the average grade attained by students 
who earn an AP score of 3 is equaled 
by students: earning a grade of “A” in 


Table 2: Correlation Coefficients of Academic Preparation Covariates 



College 

Grade 

SAT 

Math 

SAT 

Verbal 

Last 

Math 

Grade 

Last Eng 
Grade 

Science 

GPA 

Regular 

or 

Honors 

Grade 

Honors 

Calculus 

AP 

Score 

College Grade 

1.000 










SAT Math 

0.315 

1.000 









SAT Verbal 

0.200 

0.330 

1.000 








Last Math Grade 

0.317 

0.288 

0.108 

1.000 







Last Eng Grade 

0.219 

0.134 

0.200 

0.343 

1.000 






Science GPA 

0.306 

0.254 

0.190 

0.387 

0.378 

1.000 





Regular or Honors Grade 

0.269 

0.238 

0.166 

0.306 

0.265 

0.464 

1.000 




Honors 

0.128 

0.199 

0.140 

0.107 

0.098 

0.129 

0.063 

1.000 



Calculus 

0.244 

0.415 

0.203 

0.192 

0.159 

0.247 

0.196 

0.281 

1.000 


AP Score 

0.328 

0.311 

0.404 

0.216 

0.189 

0.277 

0.207 

0.004 

0.157 

1.000 
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Figure 3: Course Grade and Values of Key Covariates. 



an honors or regular science course, 
scoring a 750 on their quantitative SAT, 
who take an AP course in calculus, or 
whose last high school mathematics 
course grade was an “A.” 

Results 

Descriptive statistics 

Table 3 displays a cross tabulation 
between college science grades and the 
various academic performance levels 
displayed in Table 1 . Approximately 
13% of students in introductory 
courses did not take a corresponding 
course in high school. Roughly half of 
the students in our sample reported tak- 
ing only the regular level high school 
course in the discipline correspond- 
ing to college science course, with a 
quarter enrolling at the honors level in 
high school and 1 1 % enrolling in AP 
courses of the corresponding science. 
Of the students who reported enrolling 
in AP courses, 433 took the AP exam 
and reported their scores. 

Inf erential Analysis 


The inferential analysis offers a 
series of five multiple linear regres- 
sion models. Model A includes only 
the AP status of student: if they took 
AP, if they did not take the exam, or 
exam scores if they did take the exam. 
It explains only 2% of the variance in 
college grade. This number is to be 
expected since the AP status is only one 
factor in the background of students. 
Model B includes a dummy variable 
for each professor (each separate 
college course) and accounts for dif- 
ferences in grading severity between 
courses. This shows the predicted 
grades as if all the students enrolled 
in the same university course. This 
model shows a small upward change 
in the predicted mean grade earned by 
each AP grouping. Model C adds in 
students SAT scores, or ACT scores 
converted using a concordance table 
(Dorans, Pommerich & Lyu, 1999) 
for both mathematics and composite 
scores. Model D includes variables 
that account for general high school 
performance in science, mathematics. 


and English, typical of many studies 
accounting for high school grade point 
average (HSGPA) or its correlate, high 
school rank-in-class. Model E adds the 
kind of course and performance in any 
preparatory course prior to taking the 
AP course in the subject. This model 
seeks to control for the influence of 
different levels of preparation in biol- 
ogy, chemistry or physics. Model E 
can be interpreted as presenting the 
difference in college performance 
that is related to taking an AP course 
or to a certain performance on the AP 
test between hypothetically identical 
students in all other variables, i.e. 
between students who had the same 
preparation and background in prior 
years. It generates an estimate of the 
incremental effect of taking an AP 
course or having a certain performance 
on the AP test on the college grade in 
the same subject. Included for each 
model are the standardized beta (|3) 
coefficients that allow a comparison 
between the magnitude of the relative 
contribution of each variable to the 


Fall 2007 Vol. 16, No. 2 


11 




Table 3: Distribution of College Grades by Most Rigorous High School Course in the Subject 


College 

Science 

Grade 

Prerequisite HS Course 

Advanced 

Placement Course 


total 

not 

taken 

regular 

honors 

no 

exam 

took 

exam 

API 

AP2 

AP3 

AP4 

AP5 


55 

F 

106 

356 

117 

12 

12 

4 

3 

4 

0 

1 

603 

61 

D- 

4 

36 

1 1 

0 

3 

1 

1 

1 

0 

0 


65 

D 

102 

381 

127 

22 

11 

3 

4 

2 

2 

0 


68 

D-F 

12 

83 

21 

0 

5 

0 

3 

1 

0 

1 


71 

C- 

48 

163 

57 

5 

15 

2 

5 

5 

3 

0 

288 

75 

C 

226 

957 

382 

78 

56 

9 

17 

23 

6 

1 

1699 

78 

C-F 

46 

245 

61 

12 

22 

3 

8 

5 

5 


386 

81 

B- 

56 

225 

83 

22 

28 

7 

8 

5 

7 

1 

414 

85 

B 

243 

934 

518 

134 

79 

6 

17 

38 

12 

6 

1908 

88 

B-F 

44 

226 

102 

27 

56 

5 

16 

14 

10 

11 

455 

91 

A- 

36 

202 


22 

39 

1 

6 

15 

10 

7 


95 

A 

179 

659 

Kb 

165 

101 

4 

17 

31 

27 

22 


98 

A-F 

1 

31 


5 

6 

0 

1 

1 

2 

2 

83 

total 

1103 

4498 

2056 

504 

433 

45 

106 

145 

84 

53 

8594 

Mean 

78.79 

79.18 

81.98 

85.21 

84.35 

78.11 

82.11 

84.30 

87.15 

89.77 

80.42 

SE 

0.36 

0.17 

0.25 

0.43 

0.47 

1.63 

0.96 

0.78 

0.88 

1.05 

0.12 


explained variance. Note that the final 
regression model accounts for 35% of 
the overall variance. The lack of high 
correlation coefficients in Table 2 as 
well as other tests are evidence of a 
lack of colinearity in the models. 

A graphical comparison of Model 
A and Model E is shown in Figure 
4. (Error bars represent one standard 
error from the mean for each category.) 
The two models show the comparison 
of mean college science grades by 
simple calculation from raw grades 
in Model A and by controlling for 
the collection of important covari- 
ates in Model E. The course mean is 
represented by a horizontal dashed 
line. The magnitudes of coefficients 
for AP experience are clearly muted 
when important background factors 
not associated with AP are accounted 
for in subsequent models. The average 
slope of the data relating AP score with 
course grade is considerably reduced 
as shown in the third panel of Figure 
4. Accounting for covariates leads to 


a reduction in the magnitude of all 
AP-related coefficients. 

Model E, deemed the “full model,” 
is particularly interesting, given that 
it accounts for several factors not 
easily disaggregated from students’ 
and teachers’ experiences. This model 
includes four significant demographic 
variables that together may be thought 
of as measures of socio-economic 
status (SES). Adjustments for such 
variables can never insure that SES 
is controlled for fully, since it cannot 
be measured without error:^^ 

• Race - Accounting for the self 
identified race or ethnicity of 
students: White: Black, Hispanic, 
Asian, multi-ethnic, or other. 

• Hipared - the highest level of 
education of either parent: less or 
equal to a high school diploma, 
some college, 4-year degree, 
graduate or professional degree. 

• Hstype’-The kind of high school 
last attended: private, public or 
other. 


• Mean Ed Eevel - The average 
educational level of those in the 
student’s home ZIP code, based 
on the categories use for Hipa- 
red. 

Discussion 

The initial model which does 
not control for student backgrounds 
and academic performance prior to, 
and independent of, any AP course 
experiences, shows large differences 
between students who enroll in AP 
and others, earning college course 
grades about four points higher than 
the course average. There is little dif- 
ference between those reporting taking 
the AP exam and those who choose 
not to in AP courses. However, when 
demographics and prior academic 
achievement are accounted for, the 
apparent advantage held by students 
with AP experiences in high schools 
are roughly cut in half. This reduction 
is similar to findings by Dougherty et 
al. (2006a) in which the introduction 
of controls in their regression analysis 


23. This point suggested by an anonymous reviewer of this paper. 
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Table 4: Regression Models Explaining Variance in College Science Grade 


1 

m3 

1 Model A 

SE 

1 Model B 

SE 

1 Model C 

SE 

Beta 

1 Model D 

SE Beta 

1 Model E 

SE 

Beta 

Constant 

1 

183.79 



184.76 

* ^ 

0.45 

159.07 


1.26 


137.73 

:4c * * 

1.61 

142.84 


1.68 


iProf ID 

123 




1 


Race 

5 







* 


* * * 






hipared 

3 







* 


* * ^ 






Hstype' 

2 









* ^ ^ 






Mean Ed Level' 

1 








- 

* ^ ^ 


0.61 


0.19 

0.03 

SATM CAL' 

4 







0.01 

* * * 

0.00 

0.09 

0.01 


0.00 0.09 

0.02 

* ^ * 

0.00 

0.13 

SATV CAL' 

1 







0.03 

* * * 

0.00 

0.27 

0.01 


0.00 0.06 

0.01 

* ^ * 

0.00 

0.06 

himath 

4 










* ^ * 

0.13 




0.11 

<Algebra II 










-2.69 


0.28 

-2.31 


0.28 


precalculus 









-1.33 


0.19 

-1.16 


0.19 


Calculus 









0.36 


0.25 

0.27 


0.25 


Calculus AB 









1.38 


0.23 

1.17 


0.23 


Calculus BC 









2.28 


0.37 

2.03 


0.37 


Math Grade 

1 








2.51 

* ^ * 

0.15 0.17 

2.27 

*** 

0.15 

0.16 

English Grade 

4 










0.93 

* ^ * 

0.19 0.05 

0.75 

*** 

0.19 

0.04 

Science Grades 

1 










2.99 

* ^ * 

0.23 0.14 

2.13 

*** 

0.24 

0.10 

Prep Course 

4 















0.08 

not taken 













0.19 


0.29 


not reported 












-0.41 


0.39 


C or lower 












-1.63 


0.33 


B 












-0.28 


0.22 


A 












2.12 


0.21 


Honors 

1 



1 1 

1 1 

0.97 


0.26 

0.04 

AP Status 

6 


*** 



*** 



* * * 

0.19 

1 

0.14 


*** 


0.15 

no AP 


-3.91 


0.45 

-4.66 


0.44 

-3.57 


0.41 

-2.69 


0.39 

-2.26 


0.40 


No AP Exam 


1.42 

0.61 

0.14 


0.60 

0.07 


0.56 

0.11 


0.53 

0.16 


0.53 


AP 1 


-5.68 

1.49 

-5.22 


1.41 

-3.32 


1.33 


-2.19 


1.26 

-2.47 


1.25 


AP 2 


-1.68 

1.02 

-1.34 


0.97 

-0.80 

0.91 


-0.94 

0.86 

-1.02 


0.86 


AP 3 


0.51 

0.90 

0.78 


0.86 

0.36 

0.81 


0.18 

0.76 

0.16 


0.76 


AP 4 


3.36 

1.13 

4.22 


1.07 

2.93 


1.01 


2.22 

0.95 

2.12 


0.94 


AP 5 


5.98 

1.38 

6.08 


1.32 

4.32 


1.25 

3.31 

1.18 

3.30 


1.18 


Level of AS 


Raw Data 











Full Model 



no AP 


79.88 


7657 

80.11 


7657 

79.05 


7657 


80.18 

7575 

79.46 


7575 


No AP Exam 


85.21 


504 

84.90 


504 

82.69 


504 

82.98 

502 

81.88 


502 


AP 1 


78.11 


45 

79.55 


45 

79.30 


45 

80.68 

45 

79.25 


45 


AP 2 


82.11 


106 

83.42 


106 

81.82 


106 

81.93 

105 

80.70 


105 


AP 3 


84.30 


145 

85.54 


145 

82.98 


145 

83.05 

145 

81.88 


145 


AP 4 


87.15 


84 

88.98 


84 

85.55 


84 


85.09 

84 

83.84 


84 


AP 5 


89.77 


53 

90.84 


53 

86.94 


53 


86.19 

52 

85.02 


52 
















Total Cases 


8594 


8594 



8594 

8594 


8594 




Cases Missin 


0 



0 



0 



86 


86 




Cases Used 


8594 



8594 



8594 



8508 


8508 





















r^2= 


0.022 



0.153 



0.252 



0.336 


0.347 





resulted in reductions in the modeled 
outcome variable, college graduation 
rate, from 39% to 26% for students 
passing an AP exam. Hence, in that 
study, a third of the apparent effect of 
passing an AP exam was explained by 
the two covariates used: eighth-grade 
mathematics achievement and high 
school characteristics. Our findings 
are similar to those of Willingham & 
Morris (1986) who also found that 
half of the “AP advantage” was ac- 
counted for by matching students by 
background. 

Analyzing college grade by differ- 
ent levels of AP exam score is also 


illuminating (Figure 4). We found 
that students who are awarded an AP 
exam score of 1 earned college science 
grades no better than did those students 
who entered the college course after 
having taken a regular course in the 
subject. Similarly, students who have 
an AP exam score of 2 do no better than 
those who have completed an honors 
course. Since students usually com- 
plete either a regular or honors course 
prior to enrolling in an AP course, it 
appears as if students reporting low 
AP exam scores may have gained no 
benefit from their AP course. This find- 
ing contrasts with the belief espoused 


by many that it is always better to take 
a more rigorous course and earn a low 
grade than an easier course and earn a 
high grade (Adelman, 1999). 

Based on the results from the Full 
Model, those students who enroll in 
AP and have not taken the AP exam 
perform in college science about as 
well as APS students: about 1.5 points 
higher than the student average. AP 4 
students are predicted to earn college 
science grades 3.4 points above aver- 
age. APS students are predicted to earn 
grades 4.6 points higher than average. 
Though these predicted scores appear 
to be fairly impressive, we should con- 
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Figure 4: Comparison of Model A (“Raw Data”) and Model E (“Full Model”) 



— ' — Raw Data 

Accounting for 
Background 

Course Mean 


sider that the student average sits just 
above 80 points, amounting to about 
a grade of B-. An additional 4.6 points 
would only elevate the course grade to 
roughly 85 points, suggesting that the 
average AP 5 student in introductory 
college science would be predicted to 
earn a letter grade of B if one accounts 
for those background variables that are 
different from most students in their 
science course. 

Next we considered the relative im- 
portance of variables in the explained 
variance of the full model based on the 
standardized (3 (in units of the fraction 
of a standard deviation of college grade 
earned for one standard deviation 
change in the independent variable). 
As one might expect, AP exam score 
has a substantial role (|3 = 0.15). Yet, 
four other variables have similar rat- 
ings: high school math grade (|3 = 
0.16), SAT Math ((3 = 0.13), highest 
math course taken in HS (|3 = 0.11), 


and average science grade (in other 
subjects ((3 = 0. 10). Taken as a group, 
these are arguably stronger predictors 
of college grade than performance in 
high school courses preparatory to 
an AP course (|3 = 0.08). This may 
reflect the importance of mathemati- 
cal knowledge and reasoning ability 
for college students in the sciences 
(Burton & Ramist, 2001). 

Poor performance in AP courses 
is often attributed to a concern over 
the “lack of academic readiness” for 
advanced placement courses, particu- 
larly a deficit in the “possession of 
prior relevant knowledge (Dougherty 
et al., 2006b, p. 6).” While one may 
assume that such knowledge could be 
acquired from a preparatory course 
(e.g. regular or honors chemistry for 
an AP chemistry course), the relatively 
low standardized coefficients related to 
theses courses (0.04 for honors level 
and 0.08 for the grade in this course) 


tend to invalidate this claim. More 
important to success in college science 
appear to be general academic skills, 
particularly in mathematics. AP exam 
score had a correlation of 0.404 with 
SAT verbal scores (Table 2). This can 
be interpreted as evidence that high 
verbal functions (both in reading and 
writing) are necessary to get the most 
from an AP course (particularly read- 
ing the textbook) and to perform well 
on the AP exam. 

There are many possible explana- 
tions for why AP exam high-scorers 
do not consistently attain levels of 
performance commensurate with 
stated College Board expectations. We 
discuss some of these below: 

• Non-equivalency of the AP 
exams and college science at- 
tainment measures. AP exams 
may not fully reflect content of 
the college courses in this study, 
even though the College Board’s 
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efforts are designed to accurately 
reflect the college curriculum. 
Certainly there is content at the 
college level that is uniquely pre- 
sented hy college professors that 
cannot show up in a standardized 
AP test. Also, more goes into a 
college course grade than a single 
final exam. Students often take 
additional tests (e.g. mid-terms), 
are graded on their laboratory 
work and problem sets, and may 
be required to pursue a project or 
paper. No matter how close the 
AP exam is to the college-level 
exam, there may be missing ele- 
ments that do not make the AP 
test a valid substitute for a college 
grade in science, however similar 
it may be to the final exam. 

• Over-generosity in AP Exam 
Scoring. “In general, an AP 
grade of 3 or higher indicates 
sufficient mastery of course 
content to grant a student ex- 
emption from a college course, 
credit, or both (College Board, 
2004, p. 6.) ” The College Board 
is clear in its guidelines, adjust- 
ing the cut-points (how exam 
scores are assigned) of each of 
its AP examinations so that ”... 
the lowest AP 5 is equivalent to 
the average “A ” in college, the 
lowest AP 4 is equivalent to the 
average “B, ” andthelowestAP 3 
is equivalent to the average “C” 
(College Board, 2004, p. 11).” 
The so-called “composite score 
boundaries” set the cut-offs for 
AP exam “grades” of 1, 2, 3, 4, 
or 5 from the raw scores or “for- 
mula scores.” The setting of these 
boundaries is explained on the AP 


website.^"^ An example is given 
for the Physics C (Electricity and 
Magnetism Exam). The cut-off 
score over which students earn 
a “5” is 19.7 out of 35 problems 
correct or 56%. The cut-off score 
over which students earn a “4” is 
40% and over which they earn 
a “3” is 31%.^^ While assigning 
final “grades” is a function of item 
difficulty, such a low percentage 
of items correct translating into 
a top score is unusual for a test. 
As a criterion referenced exam, 
the criteria may be set too low 
(Eichten, 2000). 

• Lack of Sufficient Content 
Coverage in AP Exams. A 

fundamental limitation of all 
standardized exams is the amount 
of content each exam may in- 
clude. Although exams may be 
“comprehensive” in some ways, 
a single Advanced Placement 
exam has no hope of including the 
degree of comprehensive content 
coverage typically included in 
college coursework. As a result, 
AP exams are at a fundamental 
disadvantage, depending largely 
on an assessment of student 
content knowledge based on a 
sampling of student performance. 
This approach is based on the 
same premise as this large-scale 
study; however, the difference 
lies in its application. While we 
make no claims as to the perfor- 
mance of individuals based on 
our analysis, but rather limit our 
conclusions to generalizations for 
large groups, AP exams do make 
the claim of providing a highly ac- 
curate appraisal of an individual 


student’s knowledge and ability. 
The accuracy of these claims must 
be clear and unequivocal, since 
decisions for an individual are 
made on the weight of a single 
AP exam score. A decision to 
bypass an introductory course 
does have potential repercussions 
for individual students. 

• Weak Methodology in AP Score 
Validation. Eor the process of 
validation carried out by the 
College Board, items from the rel- 
evant AP exam are administered 
to college science students at the 
end of their course. Any laxity of 
effort on the part of these students 
to do well would invalidate the 
equating of AP scores to college 
grades (Eutkus, Weiss, Campbell, 
Mazzeo & Eazer 1999). The Col- 
lege Board is silent in its technical 
manual as to what motivation 
these students have to do their best 
on these exams. Also, extensive 
preparation of AP students with 
practice exams familiarizes them 
with the format and probable 
content. Participating college 
students enjoy no such “prep- 
ping” for the questions that they 
answer drawn from AP exams 
and may be unaccustomed to their 
formulation, earning them lower 
scores. There is a lack of parity 
since the College Board does not 
administer actual college science 
final exams to AP students while 
in high school to see if they score 
at levels reflective of high college 
grades. 

• Sample Size of the Current 
Study. While 8594 students were 
included in this study, only 937 


24. “An AP grade may also indicate that the student should receive credit for a college course that he or she has not taken. To check the validity of such decisions, 
studies are conducted in which the AP Exam is administered to students taking the college courses in question. The college students’ grades on the AP Exam 
are then compared with their grades in the college course.” (http://apcentral.collegeboard. com/article/0, 3045, 152-167-0-2052, 00. html#key) 

25. http://apcentral.collegeboard.com/members/article/l,3046, 154-18 1-0-1 1583,00.html 
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reported taking the relevant AP 
course in high school. Of these 
students 433 reported taking 
the exam with 282 reporting AP 
exam scores of 3 or higher. While 
this study is smaller than several 
concerning Advanced Placement, 
it is the largest investigation of 
its kind that controls for relevant 
background factors such as prior 
mathematics and science course- 
taking and performance. This 
study offers a snapshot of the 
actual performance of students 
in introductory college science 
in U.S. colleges and universi- 
ties. While studies with larger 
samples of AP students abound, 
most control for few or no back- 
ground variables. These results 
offer little to no information about 
the disaggregated relationship of 
Advanced Placement experiences 
and student performance in col- 
lege . After controlling for several 
covariates, our study found that 
AP course participation and 
performance on AP tests were 
indeed related to performance 
in college science courses, but 
that the magnitude of the effect 
was about half that found with- 
out controls. Moreover, we do 
not know if AP experiences help 
students or if better students help 
AP outcomes. Given the clear dis- 
parities of educational resources 
in American education, the latter 
may not be further from the truth 
than the former. 

• Missing students who “place 
out” in the current study. This 
study, by its very design, does not 
include any students who have 
earned high enough AP exam 
scores to receive credit for an in- 
troductory college science course 


and, hence, do not take it. Many, 
perhaps most, of the students 
in colleges that we study who 
earn a 3, 4, or 5 are simply not 
in our sample. There is no direct 
way to know how these students 
would fare in an introductory 
course, one can only analyze the 
grades of those who are actu- 
ally enrolled in an introductory 
course, as we have. However, we 
have established that these high 
scoring AP students have no ap- 
parent deficiencies in other areas. 
Through our follow-up survey we 
have also found that most thought 
retaking the introductory course 
was beneficial. These students 
strive for high grades and work 
as hard or harder than their class- 
mates. 

• AP students were bored retak- 
ing introductory courses. One 

explanation may be boredom 
among AP exam takers since 
the college course may merely 
be a review of concepts covered 
in high school. With this issue 
in mind, we queried students in 
our survey who offered “further 
contact” information and who 
reported high AP exam scores. 
Our follow-up inquiries found 
that a majority of the students 
contacted reported the “re-tak- 
ing” of the college science course 
to be beneficial. In addition, we 
did not find significant differences 
(at p<0.05 for test) in level of 
effort reported in college science 
between AP students who passed 
or failed the AP exam in high 
school. 

Conclusions 

This study seeks to disentangle 
two issues: the validity of Advanced 


Placement exams as predictors of 
college science performance and the 
“value-added” of taking AP science 
courses in high school. AP courses act 
as both a filter for the most motivated 
and brightest high school students, 
while offering an opportunity to learn 
science in a rigorous, fast-paced en- 
vironment modeling an introductory 
college course. We have chosen to 
examine the performance of college 
students taking introductory college 
biology, chemistry, and physics. 
These students have varied academic 
backgrounds that allow for the use of 
multiple regression to model the de- 
gree to which specific variables explain 
differences in college performance. 

We find that the students in our 
sample who reported AP exam scores 
of 3 or higher earned college grades 
that were higher than the student av- 
erage. However, many AP 5 students 
performed at levels below the College 
Board’s claims of excellence even 
after taking a semester of the college 
science. While 73% of AP 4 students 
earned college science grades above 
the College Board’s stated “mid-level 
B performance in college,” about 
half of the AP 5 students missed the 
A level performance cited by the 
College Board. We should be clear 
on one very important point, all the 
students in our survey experienced a 
semester of actual introductory college 
science course work. Thus, given that 
high school AP courses are intended 
to stand in place of college courses, it 
can be argued that AP students have 
taken the introductory college course 
twice, and despite this clear advantage, 
their performance is by no means 
indicative of the benefit many would 
have us believe Advanced Placement 
courses would impart to high school 
students who take them. As a measure 
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of introductory college science content 
knowledge mastery, the performance 
of a considerable fraction of AP takers 
does not appear to adequately reflect 
the professed notions of the College 
Board. In fact, in our qualitative 
probe, many students feel that while 
AP is a good preparation for college 
science, many also feel they benefited 
from taking the actual college science 
course. 

Based on our analysis, it appears 
that about half of the advantage at- 
tributed to AP experience can be ac- 
counted for by variables representing 
the academic abilities and experiences 
possessed by AP students prior to, or 
independent of, their AP course expe- 
riences. Students’ backgrounds, par- 
ticularly their mathematical and verbal 
skills, appear to contribute mightily to 
their performance in college science 
courses. While the AP examination 
program is an elaborate system, with 
its professional development program 
rivaling its assessment program in 
size and productivity, the Advanced 
Placement exams themselves appear 
to fall short of the predictive validity 
claimed by the College Board. Based 
on the findings from our study, AP 
exams scores of 3 do not appear to 
warrant the granting of college credit 
over those students who take an AP 
course in high school, but do not take 
the exam. In addition, we found that an 
AP exam score of 1 or 2 offered little 
evidence of any benefit derived from 
the AP course work experienced by the 
students in high school. In addition, 
students passing an AP exam might 
well consider retaking introductory 
courses in college to more completely 
master the content, as many colleges 
and universities already require. 



... our Study has found 
indications that the AP 
program, while certainly 
of value to many students, 
may lack some of the 
evidence necessary 
to support its claim of 
academic rigor equal to 
that of introductory college 
and university courses in 
science. 

Our findings suggest that the edu- 
cators, teachers, parents, and students 
would be better served by: 

-An acknowledgment from the 
College Board that evidence exists 
indicating the non-equivalence of 
AP courses and college courses in 
biology, chemistry, and physics. 

-A more balanced mix of both sup- 
portive and critical research at the 
Advanced Placement website.^'’ 
-The College Board clearly stating 
their role in funding research stud- 
ies cited as evidence for AP course 
effectiveness. 

-AP exam scoring made more strin- 
gent so that the highest exam score 
truly represents achievement at the 
highest level of college work. 
-The College Board acknowledg- 
ing that the later performance of 
students who pass the AP exams 
appears to be explained as much 
by their prior background as their 
AP course enrollment. 

For high schools seeking to evaluate 
their AP science courses, a preponder- 
ance of AP exam scores of 1 or 2 may 
be interpreted to mean that the AP 


course in their school offered little or 
no benefit to students beyond regular or 
honors level science courses. Schools 
in which the majority of students score 
in this range should examine whether 
exposure to AP-level rigor holds any 
benefits apart from the “academic pad- 
ding” coined by Lichten that aids them 
in the college admission process. 

Advanced Placement has become 
one of the most highly respected 
“brand-names” in secondary/post- 
secondary education. However, our 
study has found indications that the 
AP program, while certainly of value 
to many students, may lack some of 
the evidence necessary to support its 
claim of academic rigor equal to that 
of introductory college and university 
courses in science. While this current 
study certainly suffers from limita- 
tions, as do all research studies, what 
is clear from our findings is that the 
claims of the College Board regard- 
ing the interpretation and validity of 
AP exam scores are problematic. Our 
findings indicate the need for further 
investigations of greater size, scope, 
and intensity, especially given the po- 
tential being ascribed to the Advanced 
Placement program as part of the 
cure for what ails American science 
education. 
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